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Abstract 



We study generating functions for the number of permutations in S n subject 
to set of restrictions. One of the restrictions belongs to S3, while the others to 
Sk- It turns out that in a large variety of cases the answer can be expressed via 
continued fractions, and Chebyshev polynomials of the second kind. 
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1. Introduction 



Let 7T £ S n and r £ be two permutations. An occurrence of r in 7r is a sub- 



sequence 1 < i\ < i2 < ■ ■ ■ < ik < n such that (tt^ , 



. ) is order-isomorphic 



to r; in such a context t is usually called a pattern. We say that it avoids r, 
or is r-avoiding, if there is no occurrence of r in n. The set of all r-avoiding 
permutations in S n is denoted S n (r). 



Pattern avoidance proved to be a useful language in a variety of seemingly 



unrelated problems, from stack sorting [Kn 
polynomials 



[Ta] to theory of Kazhdan-Lusztig 
and singularities of Schubert varieties |LS], [3Cb]. A natural 



generalization of single pattern avoidance is subset avoidance; that is, we say 
that 7r £ S n avoids a subset T C Sk if 7r avoids any r £ T. A comp lete stud y 
of subset avoidance for the case k = 3 is carried out in f| (see also @@@). 



Several recent papers ^^ , ^^^l| ,^ g^^V| , ^V| , ^^ deal with the 
case Ti £ S3, T2 £ Sk for various pairs T\, t%. Another natural question is to study 
permutations avoiding t± and containing ti exactly t times. Such a problem for 
certain ti,T2 £ S3 and t = 1 was investigated in JR|, and for certain n £ S3, 



cw 


RWZ, 


MV1 


Ki 


,JR 


MV2 


MV3 


,MV4 



papers include continued fractions, Chebyshev polynomials, Dyck paths, and 
ordered trees. 



2 



Definition 1. A finite continued fraction with n steps define as the following 
expression 

ai 



a 2 



There are many faces for applications of theory of continued fractions as 
an examples: Theory of functions, Approximation theory Numerical analysis, 
and Restricted pattern. As an application in restricted pattern is appear the 
continued fraction 

1 



1 - x 



MV1 


Kr 


JR 


MV2, 


MV3 


MV4 



alize this continued fraction by the following. 

Definition 2. Let us denote the continued fraction with k steps 

1 



1 



1 



1-xE 



by Rk-E(x) for any k > \, and for k — we define Rq-e(x) — E. Also for 
simplicity we denote Rk-o(x) by Ru{x). 

Properties for Rk,E is given by the following proposition. 

Proposition 1. Let E any expression. Then 

(i) For all k > 1; 



Rk-E(x) = 



where Uk if the kth Chebyshev polynomials of the second kind; 
(ii) For all k > 1; 



3=1 



1 



X 2 



where Uk is the kth Chebychev polynomial of the second kind; 
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(Hi) 



lim R k . E (x) 



1 - VI - 4x 



fc-^oo "' 1 ' 2x 
Proof, (i). For k — 1 the proposition is trivial. By definitions 

f 



Rk+i-E{x) = 



1 - xRk-E(x) ' 



and by us induction we yields 

Rk+i;E(x) 



which means that 



Rk+l;E{x) = 



On the other hand, by definition of Chebyshev polynomials of the second kind 
have the following property U k {t) — 2tUf.-i(t) — U k _2(t), so 



VxU k 



( 1 



Hence the Proposition holds for k + 1 



1 



/ 1 



Again by us induction it is easy to se e the second property, and the third 
property its yield immediately from [ MV1 , Lemma 3.1]. ■ 

Example 1. By Proposition [ij-R/qo = R k (x), R kl i(x) — R k +i(x), and 
R k -i +x {x) 





- 






\fx 


u k+x 









Now, for any three set of patterns A, B, and C let us define f%B( n ) be the 
number of all a £ S n (A) such that a containing every pattern in B exactly once 
and containing every pattern in C at least once. The corresponding generating 
function we denote by F^. B {x). For simplicity, we write Fa(x), Fa-,b{x) and 
G B {x) when 5 = C = 0,C = 0, and A = C = respectively. 



The paper is organized as the following. In section 2, we find a recurrence in 
terms of generating functions for F^. B (x), and we prove F%. B (x) for all A, B, C 
such that AUB ^ is a rational function. In section 3 we present an examples of 
the main results, which are present the relation between the restricted patterns 
and continued fractions. 
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2. Main results 

Consider an arbitrary pattern t = (ti, . . . , Tfe) £ Sfc(132). Recall that Tj is said 
to be a right-to-left maximum if > Tj for any j > i. Let mo = k, mi, . . . , m r be 
the right-to- left maxima of t written from left to right. Then t can be represented 
as 

t = (r°,m ,T 1 ,TOi, . . . ,r r ,m r ), 

where each of r* may be possibly empty, and all the entries of t % are greater 
than rrii+i and all the entries of r J+1 . This representation is called the canonical 
decomposition of r. Given the canonical decomposition, we define the ith prefix 
of r by 7t 4 (t) = (r°, mo, . . . , r 4 , m^) for 1 < i < eg and 7t°(t) = t°, tt _1 (t) = 0. 
Besides, the ith suffix of r is defined by ct 1 (t) = (t 1 7 m i7 . . . ,t t ,m r ) for < 
i < r and er r+1 (T) = 0. Strictly speaking, prefixes and suffices themselves are 
not patterns, since they are not permutations (except for tt t (t) = cr°(r) = 
r). However, any prefix or suffix is order-isomorphic to a unique permutation, 
and in what follows we do not distinguish between a prefix (or suffix) and the 
corresponding permutation. Now, let us find /^"(n) in terms of /t(^) by the 
following lemma. 

Lemma 1. Let A set of patterns, and let be sequence of patterns. Then 

j=0 \ l<ii<i 2 <...<ij<m 

Proof. By definitions 

l A W+V)(' i ) = M»). 
which means this statement holds for m = 1. So more generally, 

f!" '^"<.»=i; (<-!>' x; <;:; „,<»>). 

j=0 \ l<ii<i 2 < - <ij<"i / 

by use induction and the same argument in the case m = 1, the theorem holds. 
■ 

Immediately, we can represent this result (Lemma 0) by another way, as the 
following. 

Theorem 1. Let {o/ 1 -*}™ l7 {f3^}YL i ^ e sequences of patterns such that aW 
contains /?W for all i = 1, 2, . . . , m, and Zef A set o/ patterns. Then 

m / 

/£k^w=E (-^ £ k .» 

j— \ l<ii<Z2<. ..<ij<rn 

m j 

where Ai^fy = A \J {a^}\{a^\ . . . , a<*'>} [j {/?(«}. 
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Now we present the main two results of this paper in the following two 
subsections. The first result, we find a recurrence to calculate the generating 
function Ft{x) where T set of pattern. The second result, we find an another 
recurrence to calculate the generating function Fa-b(x), which is a generalization 
of the first result. 



The generating function Fa(x). 

In the current subsection we find a recurrence to calculate Fa(x). This calcula- 
tion immediately by induction, by uses Theorem [I], and us result in [MVS, Th 
2.1]. 

Theorem 2. Let t-M = (r*' , <f; )0 , r*' 1 , d^, . . . , t*-"*, di, mi ) G S d( , (132) for i = 
1,2, ... ,p such no there two patterns one contain the another. Then 

mi m 2 m p Op-i) 



where 



such that A ilr .. d} = U w 4 il) ..., ij {7r^5 ) }UU 9 =< 1 ,..., <J {7rSV 1) }- 



The generating function i^B^)- 

Here, we find a recurrence to calculate Fa-b( x )- This calculation immediately 
by induction, by uses Theorem [y, Theorem ||, and us result in |MV3, Th 3.1]. 

Theorem 3. Let A, B any disjoint sets of pattern. Then 

(i) Fi32,A;b(x) is a rational function satisfying the relation 

x E E F^ Bi {x)F A2 , B2 {x), 

. a< =0,l,...,r Ti & 3 =0,l,...,l+r Tj 

^W-W + i E E Fl. Bi {x)F Ar ,B 2 {x), B = % 

Oi=0,l,...,r T< b;,=0,l,...,l+r T . 

where A — = 1,2,..., a}, _B = {Tj i J = 1 j 2, . . . , b}, and 
C={ 7 r a *- 1 (r i )|i = l,2,...,a}, 

A! = {7r a < (r<)|t = 1, 2, . . . , a} U ( 7j -)| j = 1, 2, . . . , 6}, 
Bi = {7r 6 *- 1 (7i)|j = l,2,...,6}, 

A 2 = {^(r,-)|i = 1, 2, . . . , a} U {^friM = 1, 2, . . . , 6}, 
B 2 = {^(7 J )|i = l,2,...,6}. 
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(ii) for two sequences of patterns {aW}^, such that contains (3^ 

for all i = 1,2, ... ,m, and for any set of patterns T , 

m 

<fcSU ;B w = El(-i) J E f Ah 

j'=0 \ l<ii <i 2 <...<ij <m 

m J 

where A^...^ =T U ■ ■ ■ , Q {/? (ld) }- 

3. Examples and continued fractions 

Though elementary, Theorem || enables us to derive easily various known and 
new results for a fixed a set of patterns. 

Example 2. (see |g[ ) An a numerical case, by Theorem || we yields 

F{ 2341,3241} fa) = 1 + ^{23,32} 2341,3241} (^) + 

23,3241 fa) ~ -F 1 {23,32}(^))-F 1 {2341,l}(a;) + 
+x(F{2341,32}(x) ~ -£{23,32} (z))-£{l,3241{ 0*0 + 

+x(F{ 2 Ul,32il}(x) ~ -£{23,3241} (x) ~ ^{2341,32} (x) + ^{23,32} (x))F{i tl y (x). 

On the other hand, by definition it is easy to see -£{23,32} (^O = F^ T ^(x) = 

1, and ^{23,3241} (x) = ^{2341,32} = t=j- Hence, it is easy to get F{ 2 341,3241} (x) = 

l—x—x 2 
l-2x-x 2 ■ 

As a corollary of Theorem |2| we obtain the following. 

Corollary 1. LetT' set of pattern, and letT = {(ti, . . ., Tfe_i, fe)|(ri, . . . , Tk—i) G 
T'}. TTien 

w 1 - xF T ' (ar) 

Example 3. (sec |S^, Pr. 15], and (g|, Sec 4.1]) An another numerical example, 
by Corollary [l] we yields 

^{123,213} (x) 



1 - xF{2i,u}{x) ' 
and by definitions we get the result |SS|, Pr. 15], which is 

/ \ 1 

^{123,213} (X) = l _ x __ x2 - 

In the same way, by use Corollary [l] twice we yields 

l — x — x 2 



-£{1234,2134} (x) 



\-2x-x 2 



which is result [|G], Sec. 4.1]. 
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Now let us generalize the above example. First let define a special set of 
patterns. 

Definition 3. For any k > I > 1, let be the set of all permutations t G S k 
such that (r; + i, ti+2, ■ ■ ■ , i~k) = (I + 1, 1 + 2, . . . , k). Clearly |C/ ; fc | = /!. 



By jRrj] and definitions 

i-i 

j=0 

where Cj is the jth Catalan number. Hence, consequentially to Example ^ we 
yields similarly the following. 

Corollary 2. Let fc > I > 1; then 

F V k(x) = R k -i;E(x){x), 

l-i 

where E(x) = ^2 CjX^ , and Cj is the jth Catalan number. 

3=0 

Example 4. (see |CW|jMVf| ,|Kj|) For I = 1, by Corollary | we yields 
F V k(x) = F 12 ... k {x) = Rk-o(x) = Rk(x). 
Now we present another direction to use continued fractions. 
Corollary 3. Let fc > I > 1 . For any r € U* , 



I 

, i 

F U*\{t};t( x ) 



X 



l-i 

where E(x) — Y c j xl , an d c j * s the jth Catalan number. 
j=o 

Proof. Let us fix t € Uf such that (r = r', fc), and let us denote U^\{t} by Mf. 
Immediately by Theorem |^ we yields 

F M*;t{ X ) = xF M*- x ;A X ) F U*{ x ) + xF uf- 1 ( X ) F Mf;r( X )' 

which means by Corollary ^| 

xR k -i- E (x) 

F M«;t( x ) = ^ ~5 TCV^Mf-^r'W' 

' 1 - XHk-l-l-E\X) ' 

2-1 

where E = Yl c i x ^ > an d c j 1S the jth Catalan number. Since 1 _ xR 1 t-jt = 

Rm;E{x) we obtain 

F M*-A X ) = xR 1-1-,e{ x ) F M^;A x )- 
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By use induction we yields 

k-l 



F M u. T {x) = x k l F M i. p {x) J] R 2 j;E (x), 



where r = (/3, l + l, 1+2, . . . , k). On the other hand, by definitions F M i.p(x) = 
so 

k-l 



F M *. T (x)=x k l[R 2 j . E (x). 



Hence by Proposition || the corollary holds. 



Example 5. (sec |MVl| ,|Ki|l) For either I = 1, E = 1; or Z = 0, E = 0, we yields 
from Corollary || for all fc > 1 

G{l2...k}(%) = F {$;12...k}( x ) 



For I = 2, E = 1 + x we obtain for all fc > 3 

-F{123...fc;213...fc}( a; ) = -F{213...fc;123...fc}(x) = 



Example 6. Now we complete all the calculation for either containing exactly 
once, or avoiding a two patterns from = {123 . . . fc, 213 . . . fc}. By Corollary 
U and Example g its left to find Gyk (x). Let fc > 3; by Theorem || 

G u k(x) = xF u k-l(x)G u k(x) + xF 12 3...k-l;213...k~l(x)F 2 13...k;12S...k(x) 
+xF213...k-l;123...k-l(x)F 12 3...k;213...k(x) + xG u k-l{x)F u k{x) , 

so by Corollary g, Example [| and by use Proposition [l] we yields 

Gc/2fc(x) = Wfc^5^(^) +xi? '- 2;1+a;(a;)G ^- l(x) ' 

where Wkufc) = u k-j(tj=) ~ xU k-2-j ( ^75) • Besides (x) = for all 
fc = 0, 1, 2, 3, hence 

Gc/ I^ = W2 ^\ £ 
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